Fuzzy Kanerva-based function approximation for reinforcement learning
نویسندگان
چکیده
Radial Basis Functions and Kanerva Coding can give poor performance when applied to large-scale multi-agent systems. In this paper, we attempt to solve a collection of predator-prey pursuit instances and argue that the poor performance is caused by frequent prototype collisions. We show that dynamic prototype allocation and adaptation can give better results by reducing these collisions. We then describe our novel approach, fuzzy Kanerva-based function approximation, that uses a fine-grained fuzzy membership grade to describe a state-action pair’s adjacency with respect to each prototype. This approach completely eliminates prototype collisions. We conclude that adaptive fuzzy Kanerva Coding can significantly improve a reinforcement learner’s ability to solve large-scale multi-agent problems.
منابع مشابه
Function Approximation Using Tile and Kanerva Coding For Multi-Agent Systems
Function approximation can improve the ability of a reinforcement learner. Tile coding and Kanerva coding are two classical methods for implementing function approximation, but these methods may give poor performance when applied to large-scale, high-dimensional instances. In the paper, we evaluate a collection of hard instances of the predator-prey pursuit problem, a classic multi-agent reinfo...
متن کاملAdaptive Kanerva-based Function Approximation for Multi-Agent Systems (Short Paper)
In this paper, we show how adaptive prototype optimization can be used to improve the performance of function approximation based on Kanerva Coding when solving largescale instances of classic multi-agent problems. We apply our techniques to the predator-prey pursuit problem. We first demonstrate that Kanerva Coding applied within a reinforcement learner does not give good results. We then desc...
متن کاملART-Based Neuro-fuzzy Modelling Applied to Reinforcement Learning
The mountain car problem is a well-known task, often used for testing reinforcement learning algorithms. It is a problem with real valued state variables, which means that some kind of function approximation is required. In this paper, three reinforcement learning architectures are compared on the mountain car problem. Comparison results are presented, indicating the potentials of the actor-onl...
متن کاملAdaptive Kanerva-based function approximation for multi-agent systems
In this paper, we show how adaptive prototype optimization can be used to improve the performance of function approximation based on Kanerva Coding when solving largescale instances of classic multi-agent problems. We apply our techniques to the predator-prey pursuit problem. We first demonstrate that Kanerva Coding applied within a reinforcement learner does not give good results. We then desc...
متن کاملExploration and exploitation balance management in fuzzy reinforcement learning
This paper offers a fuzzy balance management scheme between exploration and exploitation, which can be implemented in any critic-only fuzzy reinforcement learning method. The paper, however, focuses on a newly developed continuous reinforcement learning method, called fuzzy Sarsa learning (FSL) due to its advantages. Establishing balance greatly depends on the accuracy of action value function ...
متن کامل